An Introduction to Scoped Values in Java

After moving to the six-month release cadence, the Java language has entered a rapid development process.

While the process introduces many new features, these new features sometimes cause updates to existing APIs and sometimes result in the development of new APIs.

An example of the second is Scoped Values which has been included in the JDK since Java 20 as an incubator API.

Why were Scoped Values proposed?

Virtual threads became a part of JDK as a preview feature in Java 19.

They are a lightweight implementation of Java threads and promise dramatically reduce the effort of writing, maintaining, and observing high-throughput concurrent applications.

Virtual threads are cheap by nature.

This means that thousands or even millions of virtual threads can be used.

On the other hand, ThreadLocal API has been widely used since Java 1.2 for object sharing between application components without resorting to method arguments.

At this point, given the aforementioned nature of virtual threads, some problems arise.

What is the problem?

The ThreadLocal API supports a fully general model of communication that allows any code to mutate the data by calling related methods(eg set(), remove()) at any time, so these variables are mutable.

In many scenarios, however, Java developers need to work with immutable objects to be passed throughout a process.

The mentioned communication model of ThreadLocal API isn't conducive to such transmission, i.e., the simple one-way transmission of immutable data from one application component to another.

In addition, when a thread-local variable is written via the set() method it is retained for the lifetime of the thread, or until code in the thread calls the remove() method.

This means that per-thread data is often retained for longer than necessary.

Hence when using large numbers of threads, or when there is an inheritance relationship between the threads the overhead of thread-local variables may be even higher.

Considering this described design flaws of thread-local variables, the drawbacks of using them with tens of thousands or even millions of virtual threads are obvious.

The Scoped Values API proposed to overcome the aforementioned potential problems. They should be preferred to thread-local variables, especially when using large numbers of virtual threads.

What are Scoped Values and how to use them?

The Scoped Values API allows us to store and share immutable data for a bounded lifetime and only the thread that wrote the data can read it.

A scoped value is a variable of type ScopedValue and is typically declared as a static final field like a thread-local variable so it can easily be reached from many components.

public class PaymentGateway
{
    public static final ScopedValue<PaymentRequest> PAYMENT_REQUEST = ScopedValue.newInstance();
   
    //...
}

Once declared, a scoped value is used as shown below.

import static org.jugistanbul.PaymentGateway.PAYMENT_REQUEST;

public class PaymentProcessor
{
   public static void createPaymentTask(final PaymentRequest request){
       ScopedValue.where(PAYMENT_REQUEST, request)
                   .run(() -> PaymentService.getPaidByCreditCard());
   }
}

In the code snippet above, a scoped value and the object to which it is to be bound are passed to the where() method as a key and a value argument.

The run() call binds the scoped value to the current thread by providing a specific incarnation of it. That makes the scoped value accessible in getPaidByCreditCard() method.

In this way, notice that the where() and run() methods together provide a one-way sharing of data from one component to another.

The where() is a method of the Carrier class which is one of the inner classes of ScopedValues. It maps scoped values as keys, to values and returns a new Carrier hence the where() method can be chained.

public class PaymentService
{
   public static void getPaidByCreditCard(){
       ValidationService.checkValidity();
}

public class ValidationService
{
   public static void checkValidity(){
       PaymentRequest paymentRequest = PaymentGateway.PAYMENT_REQUEST.get();
       checkNumber(paymentRequest.cardNumber());
   }
}

The bound scoped value can be read via the value's get() method during the lifetime of the run() method, the lambda expression, or any method called directly or indirectly from that expression.

After the run() method finishes, the binding is destroyed or reverts to its previous value when previously bound, in the current thread. That is where the question of "What is the meaning of scoped?" is answered.

The value's get() call after destroyed bindings will throw an exception. You can use ScopedValue.isBound() to check if it has a binding for the current thread.

When a scoped value is written once, then is immutable, which means a caller using a scoped value can reliably pass it as a constant value to its callees in the same thread.

However, this does not mean that one callee can't share the same scoped value with a different value with its own callees in the thread. In such cases the ScopedValue API allows a new binding to be established for nested calls, this is called rebinding.

Let's say we have a service where we print the payment information after charging the payment.

We can use the current PaymentRequest instance bound to the current thread for the print process but we don't want to share sensitive information without masking it such as card number, cardholder name, etc, with the service and any method called directly or indirectly from it. This is where rebinding comes to our help.

public class PaymentService
{
   public static void getPaidByCreditCard(){
       ValidationService.checkValidity();
       getPaid();
       ScopedValue.where(PaymentGateway.PAYMENT_REQUEST, maskedPaymentRequest)
       .run(() -> PrintService.printPaymentInfo());
   }
}

The return type of the run() method is void. If the printPaymentInfo() method was returning a value, we can prefer the call() method which calls a value-returned operation to handle the returned value.

In the above code snippet, the scoped value that was initially bound in createPaymentTask() method, rebinding to a new instance of PaymentRequest in getPaidByCreditCard() method. Hence, during the lifetime of the run method, the accessible object is only this new PaymentRequest instance.

In short, the Scoped Values API doesn't allow a method body to change the binding seen by the method itself(it has no method like set()) but allows it to change the binding seen by its callees. This guarantees a bounded lifetime for sharing of the new value.

As soon as the run() call finishes in the getPaidByCreditCard() method, the binding reverts to its previous value.

How to enable cross-thread sharing?

Java developers can create their own threads for many reasons. In such a case, if the code running in a child thread needs to access the scoped value how can access it?

The answer is that use Structured Concurrency which enables cross-thread sharing.

Structured Concurrency has been included in the JDK since Java 19 as an incubator API. It treats multiple tasks running in different threads as a single unit of work.

The principal class of the API is StructuredTaskScope and scoped values are automatically inherited by all child threads created via it.

public static void getPaidByCreditCard() throws InterruptedException, ExecutionException {

    PaymentRequest request = PaymentGateway.PAYMENT_REQUEST.get();

    try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
        Future<Boolean> validation  = scope.fork(() -> ValidationService.checkValidity());
        Future<Boolean> account = scope.fork(() -> UserService.accountChecker());
            
        scope.join();
        scope.throwIfFailed();

        if(validation.resultNow() && account.resultNow()){
            getPaid();
            ScopedValue.where(PaymentGateway.PAYMENT_REQUEST, request.copyOf())
                       .run(() -> PrintService.printPaymentInfo());
        }
    }
}

The fork() method of StructuredTaskScope starts a new thread to run the given task. In the above code snippet it is called to run the ValidationService.checkValidity() and UserService.accountChecker() methods concurrently, in their own virtual threads.

StructuredTaskScope.fork() ensures that the binding of the scoped value made in the parent thread code is automatically visible to the child thread. This is an example of scoped value inheritance and it provides enables cross-thread sharing.

Because, unlike thread-local variables, there is no copying of a parent thread's scoped value bindings to the child thread, cross-thread sharing occurs with minimal overhead.

I created a repository for the scenario discussed in this article, which you can examine.

Conclusion

The Scoped Values API allows storing and sharing immutable data for a bounded lifetime.

It is recommended to be used to overcome potential problems that may arise when using thread-local variables, especially with large numbers of virtual threads.

Scoped Values must be used with Structured Concurrency to enable cross-thread sharing.

Cross-thread sharing occurs with minimal overhead because no copying of a parent thread's scoped value bindings to the child thread.

Note that Scoped Values and Structured Concurrency are still incubator APIs, so they may still be subject to fundamental changes.

References

• Scoped Values
• Virtual Threads
• Structured Concurrency